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1. INTRODUCTION 

Deep learning has become the new revolution in the arena of machine learning purposely for 
computer visualization. A deep (multilayer) artificial neural network (ANN) is taught in a controlled manner 
via back propagation and has been used vastly as computational tools to unravel difficult classification, 
regression, pattern recognition and function estimation problem as well as complicated optimization problem 
with highly non-linear and discontinuous constraints [2]. As a result of extensive studies in ANNs, has led to 
innovation of more advance type of networks such as self-organizing maps, recurrent networks, probabilistic 
neural networks, dynamic neural networks, radial basis function neural networks, complex valued neural 
networks, feed forward neural networks, genetic algorithm, fuzzy logic and wavelets . The most common 
technique applied to numerous instruments for a forward allocation of info through the neural network is feed 
forward ANNs because of the fact that method of information transmission in real and artificial neural 
networks is feed forward in nature equally and its simplicity [3]. However, a few drawbacks of ANNs are 
acknowledged as total of remarkable challenges and limitations [4]. Data multidimensional is among the 
notable limitations in supervised learning which happens when the amount of features and training points 
becomes ominously huge. Due to the resultant memory and computational necessities and the irresistible 
volume of data to practice makes ANN learning more challenging. Additional challenge in classification is 
characteristic feature overlap between different classes because the data is non-linear has made the chore of 
splitting the classes challenging. Focusing to improve on these drawbacks by utilizing heuristic method, 
Spiking Neural Network (SNNs) models of data clustering is reviewed in this paper. 
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2. CLUSTERING OVERVIEW 

Classification and clustering are both fundamental chores in data mining knowledge discovery. 
Classification techniques are used frequently in an administered learning method (supervised) while 
clustering techniques usually for unverified (unsupervised) learning in which the objective of clustering is 
descriptive; on the other hand classification is predictive [5]. To recognize the organizations in diverse data 
natures such as static, temporal and spatial temporal, many clustering method have been developed. Once the 
features values do not modify in excess of time, the static data type is identified whereas temporal data is 
considered as a series of time when the features change their value over an uninterrupted time. This is the 
classification criteria of clustering [6]: (a) Straight clustering of data against clustering of a model that is 
being learn and capture the necessary characteristic of concern from the data. (b) Clustering raw data against 
clustering of pre-processing data. (c) Clustering of dynamic, streaming data against vector based (static data) 
and (d) Clustering of complete values variable against clustering of fluctuations in the variable. 

Fundamentally, there are five major categories of clustering method for various static data (vector 
data): (a) Hierarchical approaches build the clusters by recursively separating the occurrences either a top- 
down or bottom-up manner [7]. (b) Partitioning method which assigns the centre of data points as the centre 
of the corresponding cluster. (c) Density-based methods separate the object based on the connectivity, 
boundary and density as calculate in the neighbourhood nearest point [8]. (d) Grid based clustering quantized 
the object planetary into a limited amount of cells, wherever the clustering processes is accomplished [6]. (e) 
Model based clustering targets at the finest fit data to the model by undertake a model for each cluster [9]. 
This method is one of the major approaches of Neural Network clustering. 

Corresponding with the advanced technology of computer science, the massive amount of data is 
produced. The data produced does not limit only to static data type but temporal data type is also being 
recorded by various areas such as biology, electronic, environment and also finance. This massive raw data 
recorded need a powerful analyzing tool to extract the information. Thus, cluster work for this type of data is 
also being introduced such as bi-clustering [10] that has been explored to cluster the time series gene 
analysis. This method capable to instantaneously clusters columns and row of a grid matrix together. 

The objective of this bi-clustering methods is to cluster analogous pattern into the identical group 
where it able to cope with the temporal contact among the time series data feature [10]. However, this 
method is still not capable to handle the relationship between spatio data type and temporal data type. The 
dynamic association of temporal features plays an important role that influence the training and 
transformation of pattern dynamically overtime. Thus, spatio-temporal clustering technique has been 
recommended in order to enhance the capability of clustering approach to deal with static based data and 
spatio-temporal information simultaneously based on Spiking Neural Network modelling [6]. 


3. CLUSTERING IN SPIKING NEURAL NETWORK 

Motivated by the utility and structure of how the human brain learn information where neuron 
models interconnect by structures of spikes, Spiking Neural Networks is presented. Networks collected from 
spiking neurons are capable to practice large volume of data with a moderately small amount of spikes [11]. 
Spiking networks works similarly to biological neurons can be seen in the dendrites, cell body and axon parts 
[12] as in Figure 1 which gives it the ability to solve a wide variety of definite problem in everyday 
engineering such as motor controller, event discovery, signal-processing, spatial navigating or speech 
recognition. 
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Figure |. Biological neuron model taken from [13] 
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A presynaptic neuron interconnects with a post-synaptic neuron via trains of spikes or stroke 
potentials. The transferred material is commonly programmed in the rate of recurrence of spiking (rate 
encoding) and the scheduling of the spikes (pulse encoding) [14]. Pulse encoding is extra influential than rate 
encoding in relations of the wide-ranging of info that may perhaps encoded by the identical number of 
neurons. Indeed, rate encoding can be reflected to be an exceptional instance of pulse encoding. 

The average firing rate can be figured if the spike timings are identified. Nonetheless, the spiking 
neuron has a natural dynamic nature that is categorized by an interior state which varies upon time. Every 
post-synaptic, neuron fires an action possible or spike at the time occurrence its core state surpasses the 
neuron threshold [14]. Comparable to biological neurons, the weight of the spikes does not disclose any info. 

In this paper, few applications of SNN in data clustering is presented. As an unsupervised learning, 
clustering tends to organize data samples into it homogeneous groups in which all the data samples within the 
group are similar constructed on particular designated possessions of the data [15]. Unsupervised clustering 
removes the requirement for several repetitions over a training data set which is typically obligatory for 
supervised learning procedures for example gradient lineage and its deviations [16] thus it require lower 
computational burden. Hence, many unsupervised learning applied SNNs as approached and its former 
development [3]. 

Thru adjusting synaptic strong point of Hebbian progressions lead to the redeployment of networks 
contained by a neural network and beneath certain circumstances might affect in the advent of innovative 
functions for instance input clustering, source separation, pattern recognition, dimensionality reduction, 
construction of self-organizing maps or construction of associative memories [11]. This practice usually 
implicates learning designed configurations in the data by eliminating untainted formless noise. 
Dimensionality reduction and clustering algorithms are typically unsupervised learning [17]. 


4. CLUSTERING EVALUATION 

All the clustering approach used for different clustering application such as image segmentation, 
bio-informatics of object recognition, business, gene expression data, character recognition, data mining, 
information retrieval, spatial data analysis, data reduction and big data mining has to undergo a few 
evaluation criteria to test the legitimacy and accurateness of the clusters models performed. Good clustering 
algorithms or models are considered when the models are able to handle the dimensional criterion possession 
of the data which can recognize as 3V’s concept. The concept are: (1) Volume - ability to deal with huge 
amount of data, (2) variety- capability of the clustering algorithms to grip different types of data or formats 
and (3) Velocity - The speed of the clustering algorithms on big data which are generated at high speed. The 
clusters molded by a certain scheme must be verified whether it display concentrated resemblance among the 
objects in the same cluster or slightest resemblance among those in other clusters [18]. The evaluation 
criterion is allocated into two kinds as internal quality criteria measures such as present in Table | and 
external quality criteria measures such as present in Table 2. 


Table 1. Internal Quality Criteria Measure with Their Formula 
Types Formula 
Se= DG Me) He)? 


Scatter Criteria Matrix [18]. xECK 








Where, C;, is the set of instances in cluster k; yu, is the vector mean of cluster k. 
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a a CEC Xj-XKE Cj CiEC XjECi:XpAC; 
Condorcet’s criterion is another approach Xj#Xk 
applied for the ranking problem [19]. 
Where S(X;,X,)and d(X;,X,) measure the similarity and distance of the 


vectors X; and X,. 
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The C -criterion is an extension of Geen CiEC XjEC EX RAC, 

Condorcet’s criterion [19]. dee 
Where S(X;,X,)and d(X;,X,) measure the similarity and distance of the 
vectors X, and X; ; y is the threshold value. 


K 
Sum of Square Error (SSE) is the utmost SSE = y > xi Me I|? 
regularly used criterion measure for clustering k=1 VxjECx 
[19]. 
Where C;, is the set of instances in cluster k; 4; is the vector mean of cluster k. 
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Table 2. External Quality Criteria Measure with Their Formula 


Type Formula 
TP +TN 


TP +FP+FN+TN 











RAND = 
Rand index is used to compute how similar 
the clusters are to the benchmark 


classifications (18), Where TP is the number of true positive. 


TN is the number of true negatives, FP is the number of false positives and FN is 
the number of false negatives. The Rand index lies between 0 and 1. 





Fowlkes—Mallows index defines the FM = ie . af 
resemblance between the clusters. Greater TP + FP TP+FN 
value of the Fowlkes—Mallows index presents 
higher similarity between the clusters [7]. Where TP is the number of true positive , FP is the number of false positives and 
FN is the number of false negatives. 
F-measure solve problem in Rad index where (n?+1).P.R 
the false positives and false negatives are ~ R2P+ER 
equally weighted by balancing of false 
negatives using weighting recall parameter Where P is the precision rate and R is the recall rate. Recall has no impact when 
720 [18]. n=0. 
| ANB TP 
J (A,B) = 





Jaccard index is measured to recognize the 
corresponding between two data sets [20]. 


|A UB| TP+FP+FN 


If A and B are both empty, then J (A, B) = 1. 





Internal criteria as shown in Table | are commonly used to calculate the decisiveness of the clusters 
by spreading correspondence measure methods. It calculate how closely the objects linked in a cluster by 
evaluating the cluster compactness based on variance [21]. The external value standards measure as shown in 
Table 2 can be beneficial in command to match the assembly of cluster to a predefined classification of the 
occurrences. An external index is made of consideration of prior info on the data created on several pre- 
specified structure and used as an average to legalize the clustering results. On the contrary, the internal tests 
are not reliant on prior knowledge as it inspects the clustering organization straight as of the original data. 


5. METHODOLOGY 

SNNs models contains of single or additional layers of connected neurons which are analogous to 
the ANNs structural design such as shown in Figure 2 but contrast in the neuron’s model and the type of the 
activation function [22]. Spiking neurons can be demonstrated in many diverse ways. In order to 
quantitatively define neuronal performance based on comprehensive modelling of the neuronal membrane 
potential and ion channel conductance, a number of detailed mathematical or biophysical models have been 
established. These structures of actual neurons turn out to be a central model of a concept for spiking 
neural reproductions. 
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Figure 2. Biological Neurons and Neural Network Neuron Model taken from [23] 
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The common properties of all spiking models shared with their biological counterparts based on the 
conceptual view are as followed: (1) Their possibility of firing (generating a spike) is augmented by 
excitatory inputs and reduced by inhibitory inputs, (2) their dynamics is regarded as by at least one state 
variable. The model is thought to produce one or more spikes when the internal variables of the model reach 
a certain state, and (3) they process info approaching from many inputs and yield single spiking output 
signals [11]. 

Therefore, such comprehensive networks are significant for learning the properties of numerous 
spiking configurations particularly in relations of network simulation formations. The explicitly knowledge 
on which neurons are stimulated and whether certain neurons are stimulated more often than others are 
crucial. In simplified way, SNNs are the networks of spiking neurons and SNN construction is habitually 
comparable to the traditional ANN. Figure 3 show the spiking neural networks with a few inputs, their 
connections and output obtain as spike compared with the figure of biological neurons connections. Three 
general categories spiking network topology can be classified into [11]: 
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Figure 3. A) Spiking Neural Networks and their connections versus B) Biological neurons taken from [18] 


5.1. Feed Forward Networks 

Originate mainly in areas closer to the periphery in biological neural systems. The data input to 
output units is firmly flow from one direction here but no feedback connections are present even when the 
data processing can spread over several layers of neurons as shown in Figure 4. 








input Hidden Output 
Layer Layer Layer 


Figure 4. Feed forwards networks architecture taken from [24] 


Equally, in SNN feed forward topology are typically useful to exhibiting low-level sensory systems, 
tactile or sensing olfaction. Feed forward networks are explored correspondingly in the perspective of spike 
synchronous for resolving the binding problematic grounded on spatial-temporal configurations of spikes. 
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5.2. Recurrent Networks 

At this point single neurons or populations of neurons cooperate over reciprocal (feedback) 
relations. Feedback relations effect in an inner state of the network which permits it to demonstrate dynamic 
temporal behavior as shown in Figure 5. Therefore, recurrent networks are reflected by a higher dynamics 
and theoretically complex computational capabilities compared with feed forward networks resulting in more 
challenge to control and train. 





Input Hidden Output 
Layer Layer Layer 


Figure 5. Recurrent networks architecture taken from [24] 


In order to investigate neural information processing involved in formation of associative memories 
or working memory, recurrent spiking neural networks is used. Spiking networks with recurrent relations 
have also been used to model and examine occurrences observed in the brain that arise from complex 
dynamics of reciprocal interactions for example network fluctuations or network multi-stability. In the 
competition between neurons or neural populations or signal decor relation, networks with lateral inhibitory 
communication is repeatedly used and reflected in the context of decision making. 


5.3. Hybrids Networks 

This group comprehends networks in which selected sub populations might be firmly feed forward, 
while other has recurrent topology. Communications among the sub populations may be single, directional 
or mutual. 


6. DISCUSSION 

SNNs depend on the spike timing in their learning and instigation phases which exploit time- 
missing uninterrupted stimulation functions, compared to the next generation of ANNs. Data clustering via 
Spiking Neural Networks has been an interest topic to explore recently as this method tends to solve many 
problems regarding data clustering. In order to used spikes to transfer and study the spatial/spectro temporal 
data (SSTD) that are programmed with the spiking-time actions for the temporal data and with the location of 
the synapses for the spatial data, SNNs struggle to impressionist human neurons. Here a few papers that has 
been reviewed regarding the methods of the clustering. 

First, STDP (Spike Timing-Dependent Plasticity) method state that, a synapse is depressing if the 
presynaptic spike fires after the post synaptic spike or it will be improved if it helps to induce the fire of the 
post synaptic spike [25]. This method is an improvement of Hebb’s novel work as the work does not deliver 
some earnings for decreasing the synaptic efficiency but only firming up the synapse efficacy when both the 
presynaptic neuron and post synaptic neuron are vigorous [25]. This learning function guarantees that only 
those synapses that influence the post synaptic neuron in a moment before it releases a spike are greatly 
amplified. The information for each input pattern by which interval supplies can be shown by this method. 
The results demonstrate that this model can do data clustering effectively and can extent a stable status by 
giving sufficient data testers. 

SpikeCD, spiking neural network with clustering degeneracy strategy of a parameter-insensitive 
method is proposed to improve clustering competence of SNN and increase accuracy by solving problem 
regarding Rank Order based algorithms where threshold of similarity is need to be predefined for 
construction alteration which is reflected time cost and be determined by a lot of previous understanding 
[26]. Thus, the structure adjustment process is tough to be optimized. This method combines supervised 
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learning and unsupervised clustering. The number and focal point of spiking RBF neurons are adjusted 
conferring to input data in hidden layer and are attuned by manipulating the resemblance and variance of 
training data. The number of outstanding neurons after advancement in this way is much less than the amount 
of those addition approaches produced by arbitrary correspondence and the predefined parameters. As a 
consequence, SpikeCD can succeed worthy accuracy with modest assemblies. 

Another enhancement of rank order-based learning approach is known as SpikeTemp, for SNNs 
with adaptive construction. This feed forward trained SNN contains of two layers of spiking neurons: first, an 
encoding level which temporally converts real-valued structures into spatial-temporal spike configuration. 
Second, a production level of dynamic grown neurons which accomplish the spatial-temporal classification 
[27]. This technique engaged both Gaussian approachable fields and square cosine populace training scheme 
to convert real-valued structures into spatial-temporal spike configurations [27]. SpikeTemp method tried to 
solve issues regarding the precise timing information which is being ignored and the speed of imitation time 
of the network for huge data speed and networks. It is appropriate for inputs denoted by a 1-D feature vector 
and extra proper for online schemes. Consequently, this method is computationally effective and 
supplementary appropriate for varied collection of data set. 

A novel learning procedure for spiking neural networks constructed on revision of the inner neuron 
threshold is recommended as in [28]. This threshold revision is engaged to support spiking neural neuron to 
fire the lowest amount of output spike to reserve all material of the input spike train on the equivalent period 
[28]. The inadequate presentations owed by the usage of single sensors evaluation values nonetheless the 
period of these evaluations strained to be resolve in this algorithm, but this actual time motion prediction, 
recognition and behavior mechanization may perhaps need the sensor clarifications time input along with the 
sensor values. Permitted to clustering and to vary the compound user actions, a decrypt method using SNN 
appearances is constructed. Then, the smart environment is trained for cluster user actions inside a particular 
space by exploiting the SNN ideal. This proposed technique is modest in assembly and can accomplish 
higher result of clustering [28]. 

Another approach is unsupervised method to learn and early detection of spatial-temporal patterns 
using spiking neural networks. In the paper, it was stated difficulties of learning and identifying spatial- 
temporal patterns is encountered upon demonstrating gestures or other human actions [29]. It used axonal 
conduction delays that depend on a spike-timing dependent plasticity learning regulation which learn training 
of individual shapes as sets of polychromous neural clusters. Classification is done by a similarity metric 
between the sets created on by an improved version of the Jaccard index. The method is assessed on a data 
set of hand-drawn digits. The data consist of time stamps that are captured simultaneously during the 
drawing. The outcomes demonstrate that the suggested method can effectively study these configurations 
based on ominously minor amount of training samples [29]. It also can recognize patterns before their 
accomplishment. The performance of this method is improved than or equivalent with the three other 
supervised approaches. 

Modelling of large-scale spiking neural models is a significant instrument in the mission to 
recognize brain purpose and afterward generate real-world applications. A spiking neural network simulant 
locality called HRL Spiking Simulator (HRLSim) is familiarized as an appropriate manner for employment 
on a cluster of general-purpose graphical processing units (GPGPUs). It deals with reasonable and accessible 
instrument for scheming, real-time imitation and investigating of across-the-board spiking neural networks 
[30]. This HRLSim was premeditated for both corresponding central processing unit (CPU) constructions and 
parallel general-purpose graphical processing unit (GPGPU) cluster computers. GPUs require a huge amount 
of single-instruction multiple data (SIMD) processors proficient of capably handling massive volumes of data 
in equivalent. Besides, the total correlated in creating clusters of GPUs is significantly lesser than CPU-based 
supercomputers [30]. 

Lastly, an evolving, dynamic clustering of spatial/spectro-temporal data in 3D spiking neural 
network models (eSNNs) is introduced for cluster job of spatial-temporal data conducted in the Neucube 
application of spiking neural network design [6]. The records are encrypted as spike trains before it submitted 
to the models as each spike signify a variation in the variable rate (an event). Through the unsupervised 
learning, the cluster centers were predefined by the spatial localities of the input records variables in a 3D 
SNN model. Afterwards, the clusters are developing through the learning in such they are altered nonstop 
over time imitating the dynamics of the variations in the records. During the supervised learning, clusters 
characterize the dynamic series of neuron spiking actions in a trained SNN model, detailed for a certain class 
of data or for a single occurrence [6]. The suggested clustering approaches allowed us to reasonably analyses 
the EEG data chronicled from dissimilar subject groups. 


Indonesian J Elec Eng & Comp Sci, Vol. 15, No. 3, September 2019 : 1392 - 1400 


Indonesian J Elec Eng & Comp Sci ISSN: 2502-4752 O 1399 





7. CONCLUSION AND FUTURE WORKS 

Spiking Neural Networks is proved as a significant method to solve complex time-dependent 
configuration recognition complications for the reason of dynamic illustration naturally inherent in spiking 
neural network. Theoretically show that, SNN has the skill to estimate any continuous functions. It is also 
shown that Spike-Temp and Spike-CD can classify diverse data set with better-quality of accurateness and a 
shorter simulation time is need compared to the rank order method [16]. Moreover, the features 
representation in this method can be recycled in deep neural networks to excerpt enhanced feature 
structure [15]. 

STDP methods focused more on the data clustering that encrypts the input data in temporal space 
whereas result express by HRLSim propose an actual replication instrument to learn very large-scale spiking 
neural prototypes. Likewise, clustering activity which is based on adaptive threshold of SNN and learning 
and recognizing spatiotemporal patterns method using SNN both prove that SNN can significantly handle 
spatial-temporal data type. Outcomes similarly show that regardless of the exact small amount of training 
samples, the technique has magnificently summarized the descriptive structures of each class with high 
classification grades. 

The significant purpose of clustering method implying SNN has been summarized in this paper can 
be used for a better understanding of knowledge discovery on complex data type. Future work regarding 
Spiking Neural Network clustering approach can be explored by employing the real-world data which higher 
dimensionality and temporal relation problems. Thus, the significance of SNN in solving dynamic, 
spatial/spectro-temporal data problem can be proven. 
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